{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Migrating from OpenAI to Open LLMs Using TGI's Messages API\n",
    "\n",
    "_Authored by: [Andrew Reed](https://huggingface.co/andrewrreed)_\n",
    "\n",
    "This notebook demonstrates how you can easily transition from OpenAI models to Open LLMs without needing to refactor any existing code.\n",
    "\n",
    "[Text Generation Inference (TGI)](https://github.com/huggingface/text-generation-inference) now offers a [Messages API](https://huggingface.co/blog/tgi-messages-api), making it directly compatible with the OpenAI Chat Completion API. This means that any existing scripts that use OpenAI models (via the OpenAI client library or third-party tools like LangChain or LlamaIndex) can be directly swapped out to use any open LLM running on a TGI endpoint!\n",
    "\n",
    "This allows you to quickly test out and benefit from the numerous advantages offered by open models. Things like:\n",
    "\n",
    "- Complete control and transparency over models and data\n",
    "- No more worrying about rate limits\n",
    "- The ability to fully customize systems according to your specific needs\n",
    "\n",
    "In this notebook, we'll show you how to:\n",
    "\n",
    "1. [Create Inference Endpoint to Deploy a Model with TGI](#section_1)\n",
    "2. [Query the Inference Endpoint with OpenAI Client Libraries](#section_2)\n",
    "3. [Integrate the Endpoint with LangChain and LlamaIndex Workflows](#section_3)\n",
    "\n",
    "**Let's dive in!**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "First we need to install dependencies and set an HF API key.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install --upgrade -q huggingface_hub langchain langchain-community langchainhub langchain-openai llama-index chromadb bs4 sentence_transformers torch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import getpass\n",
    "\n",
    "# enter API key\n",
    "os.environ[\"HUGGINGFACEHUB_API_TOKEN\"] = HF_API_KEY = getpass.getpass()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"section_1\"></a>\n",
    "\n",
    "## 1. Create an Inference Endpoint\n",
    "\n",
    "To get started, let's deploy [Nous-Hermes-2-Mixtral-8x7B-DPO](https://huggingface.co/NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO), a fine-tuned Mixtral model, to Inference Endpoints using TGI.\n",
    "\n",
    "We can deploy the model in just [a few clicks from the UI](https://ui.endpoints.huggingface.co/new?vendor=aws&repository=NousResearch%2FNous-Hermes-2-Mixtral-8x7B-DPO&tgi_max_total_tokens=32000&tgi=true&tgi_max_input_length=1024&task=text-generation&instance_size=2xlarge&tgi_max_batch_prefill_tokens=2048&tgi_max_batch_total_tokens=1024000&no_suggested_compute=true&accelerator=gpu&region=us-east-1), or take advantage of the `huggingface_hub` Python library to programmatically create and manage Inference Endpoints.\n",
    "\n",
    "We'll use the Hub library here by specifing an endpoint name and model repository, along with the task of `text-generation`. In this example, we use a `protected` type so access to the deployed model will require a valid Hugging Face token. We also need to configure the hardware requirements like vendor, region, accelerator, instance type, and size. You can check out the list of available resource options [using this API call](https://api.endpoints.huggingface.cloud/#get-/v2/provider), and view recommended configurations for select models in the catalog [here](https://ui.endpoints.huggingface.co/catalog).\n",
    "\n",
    "_Note: You may need to request a quota upgrade by sending an email to [api-enterprise@huggingface.co](mailto:api-enterprise@huggingface.co)_\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "running\n"
     ]
    }
   ],
   "source": [
    "from huggingface_hub import create_inference_endpoint\n",
    "\n",
    "endpoint = create_inference_endpoint(\n",
    "    \"nous-hermes-2-mixtral-8x7b-demo\",\n",
    "    repository=\"NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO\",\n",
    "    framework=\"pytorch\",\n",
    "    task=\"text-generation\",\n",
    "    accelerator=\"gpu\",\n",
    "    vendor=\"aws\",\n",
    "    region=\"us-east-1\",\n",
    "    type=\"protected\",\n",
    "    instance_type=\"p4de\",\n",
    "    instance_size=\"2xlarge\",\n",
    "    custom_image={\n",
    "        \"health_route\": \"/health\",\n",
    "        \"env\": {\n",
    "            \"MAX_INPUT_LENGTH\": \"4096\",\n",
    "            \"MAX_BATCH_PREFILL_TOKENS\": \"4096\",\n",
    "            \"MAX_TOTAL_TOKENS\": \"32000\",\n",
    "            \"MAX_BATCH_TOTAL_TOKENS\": \"1024000\",\n",
    "            \"MODEL_ID\": \"/repository\",\n",
    "        },\n",
    "        \"url\": \"ghcr.io/huggingface/text-generation-inference:sha-1734540\",  # must be >= 1.4.0\n",
    "    },\n",
    ")\n",
    "\n",
    "endpoint.wait()\n",
    "print(endpoint.status)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It will take a few minutes for our deployment to spin up. We can use the `.wait()` utility to block the running thread until the endpoint reaches a final \"running\" state. Once running, we can confirm its status and take it for a spin via the UI Playground:\n",
    "\n",
    "![IE UI Overview](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/messages-api/endpoint-overview.png)\n",
    "\n",
    "Great, we now have a working endpoint!\n",
    "\n",
    "_Note: When deploying with `huggingface_hub`, your endpoint will scale-to-zero after 15 minutes of idle time by default to optimize cost during periods of inactivity. Check out [the Hub Python Library documentation](https://huggingface.co/docs/huggingface_hub/guides/inference_endpoints) to see all the functionality available for managing your endpoint lifecycle._\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"section_2\"></a>\n",
    "\n",
    "## 2. Query the Inference Endpoint with OpenAI Client Libraries\n",
    "\n",
    "As mentioned above, since our model is hosted with TGI it now supports a Messages API meaning we can query it directly using the familiar OpenAI client libraries.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### With the Python client\n",
    "\n",
    "The example below shows how to make this transition using the [OpenAI Python Library](https://github.com/openai/openai-python). Simply replace the `<ENDPOINT_URL>` with your endpoint URL (be sure to include the `v1/` the suffix) and populate the `<HF_API_KEY>` field with a valid Hugging Face user token. The `<ENDPOINT_URL>` can be gathered from Inference Endpoints UI, or from the endpoint object we created above with `endpoint.url`.\n",
    "\n",
    "We can then use the client as usual, passing a list of messages to stream responses from our Inference Endpoint.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Open-source software is important due to a number of reasons, including:\n",
      "\n",
      "1. Collaboration: The collaborative nature of open-source software allows developers from around the world to work together, share their ideas and improve the code. This often results in faster progress and better software.\n",
      "\n",
      "2. Transparency: With open-source software, the code is publicly available, making it easy to see exactly how the software functions, and allowing users to determine if there are any security vulnerabilities.\n",
      "\n",
      "3. Customization: Being able to access the code also allows users to customize the software to better suit their needs. This makes open-source software incredibly versatile, as users can tweak it to suit their specific use case.\n",
      "\n",
      "4. Quality: Open-source software is often developed by large communities of dedicated developers, who work together to improve the software. This results in a higher level of quality than might be found in proprietary software.\n",
      "\n",
      "5. Cost: Open-source software is often provided free of charge, which makes it accessible to a wider range of users. This can be especially important for organizations with limited budgets for software.\n",
      "\n",
      "6. Shared Benefit: By sharing the code of open-source software, everyone can benefit from the hard work of the developers. This contributes to the overall advancement of technology, as users and developers work together to improve and build upon the software.\n",
      "\n",
      "In summary, open-source software provides a collaborative platform that leads to high-quality, customizable, and transparent software, all available at little or no cost, benefiting both individuals and the technology community as a whole.<|im_end|>"
     ]
    }
   ],
   "source": [
    "from openai import OpenAI\n",
    "\n",
    "BASE_URL = endpoint.url\n",
    "\n",
    "# init the client but point it to TGI\n",
    "client = OpenAI(\n",
    "    base_url=os.path.join(BASE_URL, \"v1/\"),\n",
    "    api_key=HF_API_KEY,\n",
    ")\n",
    "chat_completion = client.chat.completions.create(\n",
    "    model=\"tgi\",\n",
    "    messages=[\n",
    "        {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
    "        {\"role\": \"user\", \"content\": \"Why is open-source software important?\"},\n",
    "    ],\n",
    "    stream=True,\n",
    "    max_tokens=500,\n",
    ")\n",
    "\n",
    "# iterate and print stream\n",
    "for message in chat_completion:\n",
    "    print(message.choices[0].delta.content, end=\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Behind the scenes, TGI’s Messages API automatically converts the list of messages into the model’s required instruction format using its [chat template](https://huggingface.co/docs/transformers/chat_templating).\n",
    "\n",
    "_Note: Certain OpenAI features, like function calling, are not compatible with TGI. Currently, the Messages API supports the following chat completion parameters: `stream`, `max_new_tokens`, `frequency_penalty`, `logprobs`, `seed`, `temperature`, and `top_p`._\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### With the JavaScript client\n",
    "\n",
    "Here’s the same streaming example above, but using the [OpenAI Javascript/Typescript Library](https://github.com/openai/openai-node).\n",
    "\n",
    "```js\n",
    "import OpenAI from \"openai\";\n",
    "\n",
    "const openai = new OpenAI({\n",
    "  baseURL: \"<ENDPOINT_URL>\" + \"/v1/\", // replace with your endpoint url\n",
    "  apiKey: \"<HF_API_TOKEN>\", // replace with your token\n",
    "});\n",
    "\n",
    "async function main() {\n",
    "  const stream = await openai.chat.completions.create({\n",
    "    model: \"tgi\",\n",
    "    messages: [\n",
    "      { role: \"system\", content: \"You are a helpful assistant.\" },\n",
    "      { role: \"user\", content: \"Why is open-source software important?\" },\n",
    "    ],\n",
    "    stream: true,\n",
    "    max_tokens: 500,\n",
    "  });\n",
    "  for await (const chunk of stream) {\n",
    "    process.stdout.write(chunk.choices[0]?.delta?.content || \"\");\n",
    "  }\n",
    "}\n",
    "\n",
    "main();\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"section_3\"></a>\n",
    "\n",
    "## 3. Integrate with LangChain and LlamaIndex\n",
    "\n",
    "Now, let’s see how to use this newly created endpoint with popular RAG frameworks like LangChain and LlamaIndex.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How to use with LangChain\n",
    "\n",
    "To use it in [LangChain](https://python.langchain.com/docs/get_started/introduction), simply create an instance of `ChatOpenAI` and pass your `<ENDPOINT_URL>` and `<HF_API_TOKEN>` as follows:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='Open-source software is important for several reasons:\\n\\n1. Transparency: Open-source software allows users to see the underlying code, making it easier to understand how the software works and identify any potential security vulnerabilities or bugs. This transparency fosters trust between users and developers.\\n\\n2. Collaboration: Open-source projects encourage collaboration among developers, allowing them to work together to improve the software, fix issues, and add new features. This collective effort can lead to')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "llm = ChatOpenAI(\n",
    "    model_name=\"tgi\",\n",
    "    openai_api_key=HF_API_KEY,\n",
    "    openai_api_base=os.path.join(BASE_URL, \"v1/\"),\n",
    ")\n",
    "llm.invoke(\"Why is open-source software important?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We’re able to directly leverage the same `ChatOpenAI` class that we would have used with the OpenAI models. This allows all previous code to work with our endpoint by changing just one line of code.\n",
    "\n",
    "Let’s now use our Mixtral model in a simple RAG pipeline to answer a question over the contents of a HF blog post.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'context': [Document(page_content='To overcome this weakness, amongst other approaches, one can integrate the LLM into a system where it can call tools: such a system is called an LLM agent.\\nIn this post, we explain the inner workings of ReAct agents, then show how to build them using the ChatHuggingFace class recently integrated in LangChain. Finally, we benchmark several open-source LLMs against GPT-3.5 and GPT-4.', metadata={'description': 'We’re on a journey to advance and democratize artificial intelligence through open source and open science.', 'language': 'No language found.', 'source': 'https://huggingface.co/blog/open-source-llms-as-agents', 'title': 'Open-source LLMs as LangChain Agents'}),\n",
       "  Document(page_content='Since the open-source models were not specifically fine-tuned for calling functions in the given output format, they are at a slight disadvantage compared to the OpenAI agents.\\nDespite this, some models perform really well! 💪\\nHere’s an example of Mixtral-8x7B answering the question: “Which city has a larger population, Guiyang or Tacheng?”\\nThought: To answer this question, I need to find the current populations of both Guiyang and Tacheng. I will use the search tool to find this information.\\nAction:\\n{', metadata={'description': 'We’re on a journey to advance and democratize artificial intelligence through open source and open science.', 'language': 'No language found.', 'source': 'https://huggingface.co/blog/open-source-llms-as-agents', 'title': 'Open-source LLMs as LangChain Agents'}),\n",
       "  Document(page_content='Agents Showdown: how do open-source LLMs perform as general purpose reasoning agents?\\n\\t\\n\\nYou can find the code for this benchmark here.\\n\\n\\n\\n\\n\\n\\t\\tEvaluation\\n\\t\\n\\nWe want to measure how open-source LLMs perform as general purpose reasoning agents. Thus we select questions requiring using logic and the use of basic tools: a calculator and access to internet search.\\nThe final dataset is a combination of samples from 3 other datasets:', metadata={'description': 'We’re on a journey to advance and democratize artificial intelligence through open source and open science.', 'language': 'No language found.', 'source': 'https://huggingface.co/blog/open-source-llms-as-agents', 'title': 'Open-source LLMs as LangChain Agents'}),\n",
       "  Document(page_content='Open-source LLMs as LangChain Agents\\n\\t\\n\\nPublished\\n\\t\\t\\t\\tJanuary 24, 2024\\nUpdate on GitHub\\n\\nm-ric\\nAymeric Roucher\\n\\n\\n\\n\\nJofthomas\\nJoffrey THOMAS\\n\\n\\n\\n\\nandrewrreed\\nAndrew Reed\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\t\\tTL;DR\\n\\t\\n\\nOpen-source LLMs have now reached a performance level that makes them suitable reasoning engines for powering agent workflows: Mixtral even surpasses GPT-3.5 on our benchmark, and its performance could easily be further enhanced with fine-tuning.\\n\\n\\n\\n\\n\\n\\t\\tIntroduction', metadata={'description': 'We’re on a journey to advance and democratize artificial intelligence through open source and open science.', 'language': 'No language found.', 'source': 'https://huggingface.co/blog/open-source-llms-as-agents', 'title': 'Open-source LLMs as LangChain Agents'})],\n",
       " 'question': 'According to this article which open-source model is the best for an agent behaviour?',\n",
       " 'answer': 'According to the article, Mixtral-8x7B is an open-source LLM that performs really well as a general-purpose reasoning agent. It even surpasses GPT-3.5 on the benchmark in the article.'}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain import hub\n",
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "from langchain_community.document_loaders import WebBaseLoader\n",
    "from langchain_community.vectorstores import Chroma\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "from langchain_core.runnables import RunnablePassthrough\n",
    "from langchain_core.runnables import RunnableParallel\n",
    "from langchain_community.embeddings import HuggingFaceEmbeddings\n",
    "\n",
    "# Load, chunk and index the contents of the blog\n",
    "loader = WebBaseLoader(\n",
    "    web_paths=(\"https://huggingface.co/blog/open-source-llms-as-agents\",),\n",
    ")\n",
    "docs = loader.load()\n",
    "\n",
    "# declare an HF embedding model\n",
    "hf_embeddings = HuggingFaceEmbeddings(model_name=\"BAAI/bge-large-en-v1.5\")\n",
    "\n",
    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=512, chunk_overlap=200)\n",
    "splits = text_splitter.split_documents(docs)\n",
    "vectorstore = Chroma.from_documents(documents=splits, embedding=hf_embeddings)\n",
    "\n",
    "# Retrieve and generate using the relevant snippets of the blog\n",
    "retriever = vectorstore.as_retriever()\n",
    "prompt = hub.pull(\"rlm/rag-prompt\")\n",
    "\n",
    "\n",
    "def format_docs(docs):\n",
    "    return \"\\n\\n\".join(doc.page_content for doc in docs)\n",
    "\n",
    "\n",
    "rag_chain_from_docs = (\n",
    "    RunnablePassthrough.assign(context=(lambda x: format_docs(x[\"context\"])))\n",
    "    | prompt\n",
    "    | llm\n",
    "    | StrOutputParser()\n",
    ")\n",
    "\n",
    "rag_chain_with_source = RunnableParallel(\n",
    "    {\"context\": retriever, \"question\": RunnablePassthrough()}\n",
    ").assign(answer=rag_chain_from_docs)\n",
    "\n",
    "rag_chain_with_source.invoke(\n",
    "    \"According to this article which open-source model is the best for an agent behaviour?\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How to use with LlamaIndex\n",
    "\n",
    "Similarly, you can also use a TGI endpoint in [LlamaIndex](https://www.llamaindex.ai/). We’ll use the `OpenAILike` class, and instantiate it by configuring some additional arguments (i.e. `is_local`, `is_function_calling_model`, `is_chat_model`, `context_window`).\n",
    "\n",
    "_Note: that the context window argument should match the value previously set for `MAX_TOTAL_TOKENS` of your endpoint._\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CompletionResponse(text='Open-source software is important for several reasons:\\n\\n1. Transparency: Open-source software allows users to see the source code, which means they can understand how the software works and how it processes data. This transparency helps build trust in the software and its developers.\\n\\n2. Collaboration: Open-source software encourages collaboration among developers, who can contribute to the code, fix bugs, and add new features. This collaborative approach often leads to faster development and', additional_kwargs={}, raw={'id': '', 'choices': [Choice(finish_reason='length', index=0, logprobs=None, message=ChatCompletionMessage(content='Open-source software is important for several reasons:\\n\\n1. Transparency: Open-source software allows users to see the source code, which means they can understand how the software works and how it processes data. This transparency helps build trust in the software and its developers.\\n\\n2. Collaboration: Open-source software encourages collaboration among developers, who can contribute to the code, fix bugs, and add new features. This collaborative approach often leads to faster development and', role='assistant', function_call=None, tool_calls=None))], 'created': 1707342025, 'model': '/repository', 'object': 'text_completion', 'system_fingerprint': '1.4.0-sha-1734540', 'usage': CompletionUsage(completion_tokens=100, prompt_tokens=18, total_tokens=118)}, delta=None)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from llama_index.llms import OpenAILike\n",
    "\n",
    "llm = OpenAILike(\n",
    "    model=\"tgi\",\n",
    "    api_key=HF_API_KEY,\n",
    "    api_base=BASE_URL + \"/v1/\",\n",
    "    is_chat_model=True,\n",
    "    is_local=False,\n",
    "    is_function_calling_model=False,\n",
    "    context_window=4096,\n",
    ")\n",
    "\n",
    "llm.complete(\"Why is open-source software important?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now use it in a similar RAG pipeline. Keep in mind that the previous choice of `MAX_INPUT_LENGTH` in your Inference Endpoint will directly influence the number of retrieved chunk (`similarity_top_k`) the model can process.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index import (\n",
    "    ServiceContext,\n",
    "    VectorStoreIndex,\n",
    ")\n",
    "from llama_index import download_loader\n",
    "from llama_index.embeddings import HuggingFaceEmbedding\n",
    "from llama_index.query_engine import CitationQueryEngine\n",
    "\n",
    "\n",
    "SimpleWebPageReader = download_loader(\"SimpleWebPageReader\")\n",
    "\n",
    "documents = SimpleWebPageReader(html_to_text=True).load_data(\n",
    "    [\"https://huggingface.co/blog/open-source-llms-as-agents\"]\n",
    ")\n",
    "\n",
    "# Load embedding model\n",
    "embed_model = HuggingFaceEmbedding(model_name=\"BAAI/bge-large-en-v1.5\")\n",
    "\n",
    "# Pass LLM to pipeline\n",
    "service_context = ServiceContext.from_defaults(embed_model=embed_model, llm=llm)\n",
    "index = VectorStoreIndex.from_documents(\n",
    "    documents, service_context=service_context, show_progress=True\n",
    ")\n",
    "\n",
    "# Query the index\n",
    "query_engine = CitationQueryEngine.from_args(\n",
    "    index,\n",
    "    similarity_top_k=2,\n",
    ")\n",
    "response = query_engine.query(\n",
    "    \"According to this article which open-source model is the best for an agent behaviour?\"\n",
    ")\n",
    "\n",
    "response.response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Wrap up\n",
    "\n",
    "After you are done with your endpoint, you can either pause or delete it. This step can be completed via the UI, or programmatically like follows.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# pause our running endpoint\n",
    "endpoint.pause()\n",
    "\n",
    "# optionally delete\n",
    "# endpoint.delete()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
